Persistence of Web References in Scientific Research

The web has greatly improved the accessibility of scientific information, however the role of the web in formal scientific publishing has been debated. Some argue that the lack of persistence of web resources means that they should not be cited in scientific research. We analyze references to web resources in computer science publications, finding that the number of web references has increased dramatically in the last few years, and that many of these references are now invalid. We also find that most invalid web references can be relocated easily. We argue that, while formal references to published articles should always be used when possible, web references help to improve communication and progress in science. However, citation practices need to be improved to minimize future loss. We provide recommended practices for citing web resources, and discuss methods for relocating invalid references. The web facilitates scientific communication in many ways. Formal references to information on the web are becoming increasingly common. However, there are many invalid links on the web, leading to user annoyance and frustration. The use of web references in research articles has been of particular concern. Some have argued that Uniform Resource Locator (URL) citations should not be contained in research papers, pointing out the lack of persistence of URLs and their contents. We examine URLs contained in computer science research articles, analyzing the volume of citations, the validity of links, and the detailed nature of invalid links. We investigate URLs contained in research papers from the ResearchIndex (also known as CiteSeer) database [3, 6]. ResearchIndex indexes Postscript and PDF research articles on the web. A free service is available at http://researchindex.org/ (if this URL is invalid, try searching for ResearchIndex or CiteSeer in a search engine). ResearchIndex currently contains about 270,000 research articles, including journal papers, conference papers, and technical reports. The database represents computer science papers that are available on the publicly indexable web [5]. We analyzed 270,977 articles in the ResearchIndex database. For the 100,826 articles that were cited and linked within the database, and hence the publication year was known, we extracted all URLs (67,577 URLs), and then attempted to access each URL. Redirected URLs were followed to their new destination. The experiments were performed during May 3 May 5, 2000. URLs were extracted by searching for strings starting with (http: https: ftp:), and ending with a quote or whitespace. Trailing periods, commas, semicolons, parentheses, and brackets were removed from the strings.